Methods and systems for identifying content elements

ABSTRACT

A method of identifying content of interest in a structured electronic document by an electronic device having a processor, an input device, and a display device, includes rendering a structured electronic document to the display device; receiving through the input device at least two separate indications of content elements within the rendered structured electronic document; and identifying with the processor a common characteristic of the indicated content elements, and identifying any further content element within the rendered structured electronic document sharing the common characteristic with the indicated content elements.

The present description relates to methods and systems for identifyingcontent in a structured electronic document.

LIMITED COPYRIGHT WAIVER

A portion of the disclosure of this patent document contains material towhich the claim of copyright protection is made. The copyright owner hasno objection to the facsimile reproduction by any person of the patentdocument or the patent disclosure following publication of theapplication, as it appears in the U.S. Patent and Trademark Office fileor records, but reserves all other rights whatsoever.

BACKGROUND

It can be desirable to access content from a structured electronicdocument, for example to utilize the content for another purpose.Identifying the content within the structured electronic document sothat it can be accessed in an automated fashion, for exampleprogrammatically, can be a laborious process.

Often one goes back to the original source of the content and retrievesit from the source. In many cases the source is not available foraccess, or determining how to gain such access makes access impractical.

It is desirable to provide improved or alternate methods of identifyingsuch content. It is also desirable to provide improved or alternatemethods, tools and systems to assist with the identification of contentwithin structured electronic documents.

BRIEF DESCRIPTION OF THE DRAWINGS

Reference will now be made, by way of example, to the accompanyingdrawings which show example embodiments, and in which:

FIG. 1 illustrates an example content elements user interface screen ofa rendered version of a prior art structured electronic documentcontaining content elements for use in association with an examplecontent extraction document of FIG. 16;

FIG. 2 is an example user interface of a rendered version of a web pagebased on an example template utilizing example selected content elementsretrieve in accordance with an example content extraction document;

FIG. 3 illustrates an example embodiment of a stored content extractiondocument used in content extraction by a computer of a structuredelectronic document to fill data elements with content elements from thestructured electronic document;

FIG. 4 illustrates an example stored data structure of data elementswith relationships defined by an example content extraction document ofFIG. 3;

FIG. 5 is an example flowchart of an embodiment of a method to outputcontent of interest of a structured electronic document;

FIG. 6 is an example flowchart of an embodiment of a method to define adata structure based on content of interest from a structured electronicdocument;

FIG. 7 illustrates an example embodiment with content extraction modulein network infrastructure to extract content according to a contentextraction document of FIG. 3 to produce a user interface screen fordisplay on a display utilizing a structured electronic documentaccessible on a remote server through a network;

FIG. 8 illustrates an example browser with extensions running on acomputer for content extraction according to a content extractiondocument of FIG. 3 to produce a user interface screen for display on adisplay utilizing a structured electronic document accessible on aremote server through a network;

FIG. 9 illustrates multiple content extraction documents for multiplestructured electronic documents;

FIG. 10 is an example user interface screen for selecting elements forgrouping;

FIG. 11 is an example user interface screen with pop-up window for groupnaming;

FIG. 12 is an example user interface screen with pop-up window fornaming elements;

FIG. 13 is an example flowchart of an embodiment of a method to identifycontent of interest of a structured electronic document.

FIG. 14 is an example flowchart of an embodiment of a method to carryout element grouping;

FIG. 15 is a further example flowchart of an embodiment of a method tocarry out element grouping;

FIG. 16 is an example flowchart of an embodiment of a method to carryout array identification;

FIG. 17 is an example user interface screen for visual content elementselection;

FIG. 18 is an example flowchart of instructions within a tool to performan automated method of generating a content extraction document;

FIG. 19 is an example object model for the tool of FIG. 18;

FIG. 20 is a sample populated example object model of FIG. 18;

FIG. 21 is an example flowchart for instructions within an example toolfor automated selection ambiguity resolution for visual content elementselection;

FIG. 22 is a further example flowchart for instructions within anexample tool for automated selection ambiguity resolution for visualcontent element selection;

FIG. 23 is an example flowchart for instructions within an example toolfor visual content element selection;

FIG. 24 is an illustration of example selection ambiguity resolution inaccordance with the flowchart of FIG. 21 on an example DOM (documentobject model) tree representing an example structured electronicdocument;

FIG. 25 is an illustration of example selection ambiguity resolution inaccordance with the flowchart of FIG. 22 on a further example DOM treerepresenting an example structured electronic document;

FIG. 26 is an example object model for use in array pattern recognition;

Like reference numerals may be used in different figures to denote likecomponents.

DESCRIPTION OF EXAMPLE EMBODIMENTS

Referring to FIG. 1, an example browser page 14 displays weatherinformation for Waterloo, Ontario. Such a page 14 displays specificcontent elements 13. In this description content elements 13 arereferred to generically using reference numeral 13. A specific contentelement may be referred to using reference numeral 13 followed by aletter, for example 13 a.

Referring to FIG. 2, similar content elements 13 are displayed on analternate page 800.

Referring to FIG. 3, a common expression 3 is stored in a computerreadable storage medium 2. A data structure definition 4 is also storedon a computer readable storage medium 2. The common expression 3identifies one or more content elements 13 in a structured electronicdocument 15. A structured electronic document is an electronic documentstructured in accordance with one or more structured definitionlanguages, such as HTML (Hyper Text Markup Language), XML (ExtensibleMarkup Language), or WSDL (Web Service Definition Language). Thestructured electronic document 15 can, for example, be used to renderpage 14 (see FIG. 1). The data structure definition 4 defines a datastructure 49. The data structure definition 4 is defined according tothe common expression 3. The common expression 3 and data definition 4can be used together to extract content elements 13 from a structuredelectronic document 15. Extracted content elements can be used, forexample, to render a page 800 (see FIG. 2).

A common expression 3 and data structure definition 4 can be utilized tooutput content of interest to a first structured electronic document 15from a second structured electronic document 15. The second structuredelectronic document 15 can, for example, be another instance of thestructured electronic document 15 used to form the common expression 3.For example, the structured electronic document 15 from which the commonexpression 3 is formed can be a weather page for an earlier date,whereas the second structured electronic document 15 can be a weatherpage created at the time the common expression 3 is to be applied.

It is recognized that the common expression 3 and data structuredefinition 4 can be stored together on one medium 2, or distributedacross different storage media 2. The common expression 3 and datastructure definition 4 can be stored in a single file.

The common expression 3 and data structure definition 4 can togetherform a content extraction document CED 1 to extract content elements 13from a structured electronic document 15, which CED 1 can be stored inthe same file in the storage medium.

For ease of reference, this description will primarily describe a CED 1embodiment; however, it is understood that the description also appliesto distributed embodiments of a common expression 3 and data structuredefinition 4.

CED 1 stored on a computer-readable medium 2 has a common expression 3and a data structure definition 4. The data structure definition 4 caninclude a data element identifying section 6 identifying data elements 5for the identified content elements 13, and a relationship definitionsection 7 defining relationships between the identified data elements 5.

An electronic device 11 has processor 1100, input device 1102, anddisplay device 1104. The input device 1102 can include, for example, apointing device 1106, such as a mouse, track ball, tablet, joystick orother pointing device. As a further non-traditional example, contentelements 13 could be highlighted on the rendered document 15 with aunique reference such that the content elements 13 can be selectedutilizing a microphone and voice recognition software operating on theelectronic device 11. For example, various content elements 13 could beidentified by unique references 1, 2, 3 etc. Then the third contentelement 13 could be chosen by speaking “3” and speaking the uniquereference to the microphone. Keyboard 1108 may be similarly used toindicate a particular content element 13. The input device 1102 can alsoinclude a keyboard 1108. The keyboard 1108 may act as a pointing device1106 by highlighting and indicating selection of a content element 13for example. The processor 1100 may be, for example, any processingdevice configured to carry out the functions described herein for thedevice 11. Example processing devices can include a suitably programmedmicroprocessor, or an appropriately configured application specificintegrated circuit (ASIC). The electronic device 11 can be, for example,a personal computer, or other computing device. The electronic device 11can include memory 1110, typically random access memory (RAM) within theprocessor 1100 to store any applicable computer programs for execution.Such memory 1110 may be further used to store other components describedherein, such as the common expression 3 and the data structuredefinition 4, and the data structure 49. Alternatively, memory 1112external to the processor 1110 and internal to the electronic device 11or memory 1114 externally accessible to the electronic device 11 may beutilized. Computer-readable storage medium 2 can form the memory 1110,112, or 114. Alternatively, the computer accessible medium 2 can belonger term storage such as a hard drive, CD-ROM disk, or flash memory.It is recognized that some forms of flash memory can be used as both RAMand long term storage.

Referring to FIG. 4 as an example related to weather, the CED 1 canidentify data elements 5 which correspond to content elements 13. Forexample, icon 5 a representing weather conditions, and alphanumericweather temperature 5 b, conditions 5 c, temperature units 5 d, andlocation 5 e correspond to respective content elements 13, such as icon13 a, temperature 13 b, conditions 13 c, temperatureUnits 13 d, andlocation 13 e. Further data elements 5 could include a short termweather forecast 5 g, and additional weather condition icon 5 h,conditions 5 i, temperature 5 j, day 5 k, and time 5 m corresponding toicon 13 h, conditions 13 i, temperature 13 j, day 13 k and time 13 m.

The CED 1 can define a relationship between the identified data elements5, for example, by grouping them together in a group data element 21,implying a common characteristic between the data elements 5 within thegroup 21 for example, implying that the icon 5 a represents the weatherconditions 5 c with a temperature 5 b for the location 5 e with shortterm weather forecast 5 g. A further relationship between the identifieddata elements 5 can be defined by defining short term weather forecast 5g as containing a further group 23 a of icon 5 h, conditions 5 i,temperature 5 j, day 5 k, and time 5 m, for example, implying that forthe short term weather conditions are represented by the icon 5 h withtemperature 5 i on the day 5 j at the time 5 k. Other aspects of thedefined relationship can include, for example, that the short termweather 5 g can contain multiple versions 23 a, 23 b, 23 c of the groupthe version contains. For example, there can be a short term weathercollection 23 a for a morning time 5 k of the given day 5 j, acollection 23 b for an afternoon time 5 k, and a collection 23 c for anevening time 5 k.

Referring to FIGS. 3, and 4, a data structure 49 can be defined toinclude data elements 5 that correspond to respective content elements13 within a structured electronic document 15. The data structuredefinition 4 can be defined according to the common expression 3 as thecommon expression 3 identifies respective content elements 13 within astructured electronic document 15. In this description data elements arereferred to generically using reference numeral 5. A specific dataelement may be referred to using reference numeral 5 followed by aletter, for example 5 a. The data elements 5 can be stored in the datastructure 49. The data structure 49 can include the entirety of theidentified data elements 5, 21 and the relationships between the dataelements 5.

Referring again to FIG. 3, the sections 3, 6 and 7 have been illustratedas separate sections for ease of comprehension. It is to be understoodthat the sections 3, 6 and 7 can be merged into one or more sections.Further, functions of the sections can be explicit or implicit, suchthat identification or definition can be expressly made, or can beimplied by other expressions. As an example, defining relationshipsbetween elements can impliedly identify those elements. Also, sectionscan be split into further sections with the functions of section 3, 6and 7 performed by more than one section. It is understood that each ofthese alternatives is encompassed within the illustration of thesections as sections 3, 6 and 7 in FIG. 3.

As part of a common expression 3 for an electronic device 11 to identifycontent elements 13 within a structured electronic document, the CED 1can identify the location of the structured electronic document 15. Forexample, the CED 1 can specify a URL (Uniform Resource Locator) for thedocument 15. Alternatively, the document 15 can be dynamic, such that,an action needs to be carried out at a first structured electronicdocument (such as a search request page) in order to generate astructured electronic document 15 (such as a search results page). Therequired action at the first structured electronic document can bedefined in the CED 1, along with all necessary inputs to the firststructured electronic document. Alternatively, some structuredelectronic document 15 hosts may allow inputs to be included in a URL tobypass a first structured electronic document.

As part of a common expression for an electronic device 11 to identifycontent elements 13 within a structured electronic document 15, the CED1 can identify the locations of the content elements 13 within thestructured electronic document 15.

Referring to FIG. 3, in order to identify respective content elements 13within a structured electronic document 15 a path can be used. A path ismade up of a sequence of path parts. Each path part may be considered astep. Within each step is an element and optionally an index if theelement is an array element. A path part can also have an identifierthat is included after an element name after a number sign or hash (“#”)or other suitable separator. A path root is the highest level elementfrom which the steps begin. The highest element can be the leftmostelement. In the examples used herein, the steps are separated by aforward slash (“/”). Steps at the same level are herein referred to ascorresponding steps. Solutions can be implemented employing otherseparators.

As an example path, a content element 13 path can be structured as anXPath. For example the path /bookstore/book/title might specify alltitles for books on a bookstore page. Where bookstore is a root elementand book and intervening element in the structured electronic document15.

The content elements 13 within the structured electronic document 15 canbe identified, for example, using paths based on the XML Path Language(XPath) (http://www.w3.org/TR/xpath), commonly referred to as an XPath.The XML Path Language is a language defined by the World Wide WebConsortium for selecting nodes from an XML document. It is to berecognized that XPath expressions are typically used in queries toreturn a result set; however, expressions using an XPath syntax are usedherein to identify content elements 13 within the structured electronicdocument 15.

Referring to FIGS. 3 and 4, relationships between the data elements 5 inthe data structure 49 can be defined based upon the structure of pathsidentifying content elements 13 within the structured electronicdocument 15.

Referring to FIG. 5, to output content from a structured electronicdocument 15, at 820 common expression 3 and a data structure definition4 are loaded into memory 1110. The common expression 3 identifies acontent element 13 in a first structured electronic document 15. Thedata structure definition 4 is defined according to the commonexpression. At 822, a data structure 49 is created in memory 1110 usingthe data structure definition 4.

At 824, the common expression 3 is applied with a processor 1100 to asecond structured electronic document 15 to extract a content element 13from the second structured electronic document 15.

At 826, the extracted content element 13 is stored in that datastructure 49.

At 828, a template structured electronic document 57 (see FIG. 7) ispopulated using the data structure 49 to produce an output structuredelectronic document 800 (see FIG. 2).

Referring to FIG. 6, at 801, content elements 13 can be indicated byreceiving through an input device 1102 an indication of a plurality ofcontent elements 13 within a structured electronic document 15. At 802,the portions can be determined with the processor 1100 from indicatedcontent elements 13. Determined portions can be, for example, individualcontent elements 13 or arrays of content elements 13.

At 804, the common expression 3 can be formed with processor 1100 basedon determined portions of the structured electronic document 15associated with each content element 13. When the common expression 3 isapplied to another structured electronic document 15, another contentelement is extracted from the other structured electronic document.

At 806, a data structure 49 can be defined according to the commonexpression 3. The data structure 49 can be further defined by a datastructure definition 4 that is defined according to the commonexpression 3.

Referring to FIG. 7, content extraction according to the CED 1 documentcan be performed by remote server 51 through a content processing module53 that performs functions similar to the browser extension 41 to carryout the retrieval of the structured electronic document 15, parsing ofthe CED 1 and the document 15, filling of the data structure 49, andformatting for presentation, with the assistance of presentation tier42, prior to transmission to the browser 40. Typically the developerportion of the program 39 described earlier will remain as an extensionto a browser 40 as the developer will likely be working at an electronicdevice 11 that is a desktop computer in order to perform developmentfunctions. Users of the additional structured electronic documents, asopposed to the developer, can utilize other forms of electronic device11 without extension 41.

The network infrastructure can be accessible to the electronic device 11through a public or private network 48 a. Similarly, the remote server47 can be accessible through a public or private network 48 b. Thenetworks 48 a, 48 b can be the same network, and can also be acombination of private and public networks. Content extraction accordingto the CED 1 at the handheld infrastructure 51 is particularly wellsuited to use in association with wireless handheld devices aselectronic device 11 by reducing quantity of data transmitted to thewireless network device, and off-loading the processing requirements tothe remote server 51. It is also to be recognized that the contentextraction by a content processing module 53 according to the CED 1 tocreate and fill a data structure 49 and provide the data structure 49 topresentation tier 42 could occur at the remote server 47 or anotherremote server or be distributed across remote servers. For example,corresponding CEDs 1 could be stored at the remote server 47 for themarkup documents 15 stored at the remote server 47; so that, a websiteprovider can provide parallel websites for different browserconfigurations, such as one using the structured electronic documents 15for an electronic device 11 that is a desktop computer and anotherproviding handheld browser optimized user interface screens using a CED1.

A listener 59 in the remote server 51 can listen for a request toperform content extraction according to CED 1 or presentation template57. If such a request is received, the listener 59 can begin theinitiation process for the template 57 or CED 1. For example, a requestmight be inferred by the listener 59 where a CED 1 or presentationtemplate 57 is associated with a particular URL requested by a browser40. For example, the listener 59 can listen for a request on an HTTPover TCP/IP (HyperText Transport Protocol over Transmission ControlProtocol/Internet Protocol) transport layer. The listener 59 may alsolisten on other transport protocols.

The remote server 51 may also allow the initiation of content processingaccording to the CED 1 by a user and presentation of the contentelements 13 to a user through transport protocols other than HTTP(HyperText Transport Protocol), such as email (SMTP) or other messagingprotocols such as SMS (Short Message Service). A combination oftransport protocols can be used, for example initiation by HTTP andpresentation by SMTP. Automatic initiation of a CED 1 is also possible.For example, a CED 1 can be initiated automatically once per day to sendout a weather report by email.

By default, content elements 13 may be transmitted to the address fromwhich the request is received. As part of the registration of a CED 1, aremote server can also receive user choices for desired protocols onwhich to listen for requests and on which to transmit content elements.For example, an email address can be associated with the CED 1 such thatthe content elements 13 are transmitted to the email address aftercontent extraction according to the CED 1.

The content processing module 53 can include a markup language parser 65(typically an HTML parser for parsing HTML for most documents 15) and acontent extraction engine 61. First the parser 65 parses the receivedstructured electronic document 15 and generates a DOM (document objectmodel) 63. The DOM 63 may be represented in many ways as is known tothose of skill in the art. In this description the DOM 63 will begraphically represented as a tree structure. The DOM 63 is processed bythe content extraction engine 61. The content extraction engine 61processes the DOM 63 according to the CED 1 to create a data structure49 in accordance with the data structure 49 definition in the CED 1. Thecontent extraction engine 61 then populates the data structure 49 withthe associated content elements 13 identified in the CED 1. The contentextraction engine 61 traverses the paths of the DOM 63 specified in theCED 1 to find the specified content elements 13. The content extractionengine 61 then populates the data structure 49 with the content elements15 in accordance with the associations in the CED 1.

Relationships between data elements can be defined by defining the datastructure 49 for the elements. A data structure definition 4 can includea path wherein a separator between data elements 5 indicates that a dataelement 5 to the right of the separator is nested within a data element5 to the left of the separator, and wherein an array representationindicates that a data element to the left of the array representationcontains an array of data elements. For example, a data structure 49 canhave paths such as: responseElement. ShortTermDay [x]. temperature.Elements are separated by a “.” (dot). A “.” (dot) indicates that theelement to the right is nested within the element to the left and “[x]”indicates that the element to the left has an array of elements. Forexample, shortTermDay has an array of ShortTermDay elements, one arrayrecord for different times of a day (for example, morning, afternoon,evening). A rResponseElement is a top-level container data element ofthe data structure 49. The responseElement contains the other dataelements.

Relationships between data elements can be defined through the use ofXML schema. Data elements can be defined within an XML schema. Using XMLschema, data elements can be combined into arrays and complex types.Additionally, data types can be defined for data elements.

As an example, structured electronic document 15 can contain hypertextmarkup language (HTML) content containing HTML content elements 13. HTMLcontent can be utilized from, for example, a web page by treating theHTML as a data source. Other markup language formats can be used, suchas, for example, the XML, Extensible Hypertext markup Language (XHTML)and others.

The Web Services Definition Language (WSDL) can be used as the basisformat for a CED 1. Although WSDL was originally created for webservices, WSDL is extensible and can be used for other purposes. WSDL issuited for use with a CED 1 as WSDL allows the definition of a serviceto obtain a structural electronic document 15, is extensible toincorporate the use of other structured definition languages. WSDL is astandard, and there are various WSDL processing tools available.

A WSDL CED 1 can be made up of a plurality of sections. Examples ofthese sections will now be described for an example CED 1 used withreference to a structured electronic document which displays on abrowser page 14 of a display 1104 as shown in FIG. 1 in response to arequest for weather information for Waterloo, Ontario.

A WSDL types section contains an XML Schema (XS) that definesrelationships between data elements using complex types. Continuing withthe weather example of FIG. 1, an example WSDL types section with XMLschema relating various named data elements to a top levelresponseElement data element to define a data structure 49 (providing adata structure definition 4) can include:

<wsdl:types> <xs:schema targetNamespace=“test.net.rim”elementFormDefault=“qualified”> <xs:element name=“responseElement”><xs:complexType> <xs:sequence> <xs:element name=“icon”type=“xs:string”/> <xs:element name=“temperature” type=“xs:string”/><xs:element name=“conditions” type=“xs:string”/> <xs:elementname=“temperatureUnits” type=“xs:string”/> <xs:element name=“location”type=“xs:string”/> <xs:element name=“shortTermDay”maxOccurs=“unbounded”> <xs:complexType> <xs:sequence> <xs:elementname=“icon” type=“xs:string”/> <xs:element name=“conditions”type=“xs:string”/> <xs:element name=“day” type=“xs:string”/> <xs:elementname=“time” type=“xs:string”/> </xs:sequence> </xs:complexType></xs:element> </xs:sequence> </xs:complexType> </xs:element></xs:schema> </wsdl:types>.

Referring to FIGS. 4 and 1, the data structure 49 responseElementdefined in the XML schema includes a collection of elements, such asicon 5 a, temperature 5 b, conditions 5 c, temperatureUnits 5 d, andlocation 5 e, which correspond in FIG. 1 to the respective contentelements 13, such as icon 13 a, temperature 13 b, conditions 13 c,temperatureUnits 13 d, and location 13 e. The responseElement alsodefines a ShortTermDay data element 5 g that is a group that includes afurther collection of elements, such as icon 5 h, conditions 5 i,temperature 5 j, day 5 k and time 5 m, corresponding to the icon 13 h,conditions 13 i, temperature 13 j, day 13 k and time 13 m elements inFIG. 1.

A common expression 3 to identify content elements 13 within astructural electronic document 15 is provided in the remainder of theWSDL.

A WSDL message definition section defines an input message named “SearchRequest” and an output message named “Search Response”. TheSearchRequest message contains the marked form elements as parts, andthe SearchResponse message contains the responseElement data structure49 as a part. Continuing with the weather example of FIGS. 4 and 1, anexample WSDL message section can include:

<wsdl:message name=″SearchRequest″> <wsdl:part name=”requestElement”></wsdl:message> <wsdl:message name=″SearchResponse″> <wsdl:partname=″responseElement″ element=″impl:responseElement″/> </wsdl:message>.

A WSDL Port Types section defines a port type (SearchInterface) with anoperation (Search) to be carried out using the input messages and theoutput messages. Continuing with the weather example of FIGS. 4 and 1,an example WSDL Port Type section can include:

<wsdl:portType name=“SearchInterface”> <wsdl:operation name=“Search”><wsdl:input message=“impl:SearchRequest”/> <wsdl:outputmessage=“impl:SearchResponse”/> </wsdl:operation> </wsdl:portType>.

A WSDL Binding section defines a binding (Binding) of the port types(SearchInterface) with the operation (Search) and the input messages(SearchRequest) and the output messages (SearchResponse). Alsoassociated with the binding is an HTML binding (an extension element) ofcontent elements 13 paths to the respective data element 15 of the datastructure 49 response element. The bindings provide an associationbetween the data elements 5 in the data structure 49 and the contentelements 13 in the document 15. Continuing with the weather example ofFIGS. 4 and 1, an example WSDL Binding section can include:

<wsdl:binding name=“Binding” type=“impl:SearchInterface”><wsdlsoap:binding style=“document”transport=“http://schemas.xmlsoap.org/soap/http”/> <wsdl:operationname=“Search”> <wsdl:input name=“SearchRequest”> <wsdlsoap:bodyuse=“literal”/> </wsdl:input> <wsdl:output name=“SearchResponse”><wsdlsoap:body use=“literal”/> <html:htmlBinding> <html:nodeBindinghtmlPath=“/div#outer_wrapper/div#content_wrapper/div#pagetitle/div#submasthead/H1/text[1]” bindingPath=“responseElement.location”/><html:nodeBindinghtmlPath=“/div#outer_wrapper/div#content_wrapper/div#obs/div#obs_conds_wrap/div#obs_conds/img/src” bindingPath=“responseElement.icon”/><html:nodeBindinghtmlPath=“/div#outer_wrapper/div#content_wrapper/div#obs/div#obs_conds_wrap/div#obs_conds/div#obs_currtemp/div/text[1]”bindingPath=“responseElement.temperature”/> <html:nodeBindinghtmlPath=“/div#outer_wrapper/div#content_wrapper/div#obs/div#obs_conds_wrap/div#obs_conds/div#obs_currtemp/div/text[3]”bindingPath=“responseElement.temperatureUnits”/> <html:nodeBindinghtmlPath=“/div#outer_wrapper/div#content_wrapper/div#obs/div#obs_conds_wrap/div#obs_conds/div#obs_currtemp/text[2]”bindingPath=“responseElement.conditions”/> <html:nodeBindinghtmlPath=“/div#outer_wrapper/div#content_wrapper/ul#precip_data/li#rain/text” bindingPath=“responseElement.rain”/> <html:nodeBindinghtmlPath=“/div#outer_wrapper/div#content_wrapper/ul#precip_data/li#rain/li#snow/text” bindingPath=“responseElement.snow”/> <html:nodeBindinghtmlPath=“/div#outer_wrapper/div#content_wrapper/div#stermfx/table/tr[0]/td[x]/img/src” bindingPath=“responseElement.shortTermDay[x].icon”/><html:nodeBindinghtmlPath=“/div#outer_wrapper/div#content_wrapper/div#stermfx/table/tr[1]/td[x]/text”bindingPath=“responseElement.shortTermDay[x].conditions”/><html:nodeBindinghtmlPath=“/div#outer_wrapper/div#content_wrapper/div#stermfx/table/tr[2]/td[x]/text”bindingPath=“responseElement.shortTermDay[x].temperature”/><html:nodeBindinghtmlPath=“/div#outer_wrapper/div#content_wrapper/div#stermfx/table/thead/tr/th[x]/text” bindingPath=“responseElement.shortTermDay[x].day”/><html:nodeBindinghtmlPath=“/div#outer_wrapper/div#content_wrapper/div#stermfx/table/thead/tr/th[x]/text[1]”bindingPath=“responseElement.shortTermDay[x].time”/> </html:htmlBinding></wsdl:output> </wsdl:operation> </wsdl:binding>.

With the operation defined in the binding section, a binding fills ininformation about the inputs, adding in the HTTP Method of the form(either GET or POST) through proprietary tags.

Binding paths represent a hierarchy in which the output node (dataelement 5) is stored within the WSDL binding. Binding paths map HTMLpaths (content element 13 path) to internal names, which can look like:responseElement. ShortTermDay[x].temperature. Again, a “.” (dot)indicates that the item to the right is nested within the item to theleft and “[x]” (an x within square brackets) indicates that the outputnode contains an array of elements.

Lastly, a service is defined which associates the binding with a port.Within the port, the address is set to the action attribute of themarked input form, or if no input form was marked, it is set to the URLof the output page.

A WSDL Service section defines a service (Service) and a port (Port) forthe service. A specific resource for obtaining the structured electronicdocument is associated with the Port by listing an HTML reference withinthe WSDL Port definition. Continuing with the weather example of FIGS. 4and 1, an example WSDL Service section can include:

<wsdl:service name=“Service”> <wsdl:port name=“Port”binding=“impl:Binding”> <html:address location=“(www.example.com)”/></wsdl:port> </wsdl:service>where “www.example.com” is the url of the structured electronic document15 or an input form required to obtain the structured electronicdocument 15. Necessary inputs, such as, for example, the desiredlocation for the weather can be specified as part of the URL if acceptedby the host in order to bypass an input form.

It is to be noted that throughout the above examples the expression“html:” is a reference to a non-WSDL element that is not part of theWSDL standard. XML Schema references are non-WSDL elements that are partof the WSDL standard. Namespaces for these can be referenced at theoutset of the CED 1 so that the WSDL parser will know what action totake.

In order to extract content according to a CED 1 in the WSDL format with“html:” and “xml” references, a WSDL parser could initially generate aWSDL structure, including the data structure 49 from the XML references.The structured electronic document 15 can be retrieved and passed to anHTML parser 65 to create an HTML DOM (document object model) 63. The DOM63 can then be processed in accordance with the paths specified in thebinding to obtain the content elements 13 specified by the paths. Thecontent elements 13 can then be used to fill the data structure 49specified in the WSDL structure in accordance with the bindings.

In order to recognize path structure to obtain content elements 13appropriate computer-readable instructions can be executed to processthe paths to obtain the content elements and to fill the data structurein accordance with the bindings. The instructions will depend on thespecific implementation of the path language chosen and the environmentwithin which the instructions are to be run. For example, theinstructions could be provided as a separate program from the HTMLparser. As an alternative, the instructions, or a portion thereof, forpath parsing can be incorporated into the HTML parser 65 such that theresulting HTML DOM 63 is expressed only in terms of the specified pathlanguage. This can result in a much smaller HTML DOM that can beprocessed more quickly to find the content elements 13 to populate theWSDL structure.

The generation of such computer-readable instructions will be within theskill of a person skilled in art utilizing their common generalknowledge and the information contained in this disclosure.

After the data structure 49 is filled it can then be used in conjunctionwith various presentation tiers, including associated presentationtemplates 57 for the structured electronic document 15. The templatescan be associated to the data structure 49 by the URL of thepresentation template 57. Example presentation tiers 42 and presentationtemplate 57 include those used, for example, in association withJavaServer Pages (JSP) and Java Server Faces (JSF). JavaServer,JavaServer Pages and JavaServer Faces are trademarks of SunMicrosystems.

Although reference is made herein to examples utilizing HTML, theprinciples described herein are not limited to HTML. Also, the WSDLexamples are similarly not limited to WSDL. Embodiments include the useof any computer-readable markup language. HTML has been used as anexample as many computer-readable structured electronic documentsutilize some form of HTML.

An example tool implemented for example as an extension 41 to a browser40, or independently therefrom, will be described. The tool providesautomated generation of a CED 1 based upon user selection of contentelements 13 from a rendered structured electronic document 15.

The tool provides visual content element selection, pattern recognitionfor array path conversion and path merging, grouping, and automatedgeneration from identified content elements 13 of a data structuredefinition 4 with associated identification of content elements 13within a structured electronic document 15. The data structuredefinition 4 with associated identification of content elements 13 areembodied within a WSDL document; however, it is to be recognized thatthe definition 4 and associated identification could be embodied inother formats, such as a data structure 49 with associatedidentification of content elements 13 that are used to populate the datastructure 49. In this case, the data structure 49 defines a copy ofitself.

Referring to FIGS. 3 and 8, the electronic device 11 can extract contentaccording to the CED 1 to obtain content elements 13 from the structuredelectronic document 15. The common expression 3 and data structuredefinition 4 can be loaded into memory 1110, 1112, or 1114. An instanceof the data structure 49 defined by the data structure definition 4 canbe created in memory 1110, 1112, or 1114. The processor 1100 can applythe common expression 3 to a structured electronic document 15 toextract one or more identified content elements 13 from the structuredelectronic document 15 to extract one or more identified contentelements 13 from the structured electronic document 15. Each extractedcontent element 13 is stored in the instance of the data structure 49. Atemplate structured electronic document 57 (see FIG. 7 and relateddescription) can be populated from the instance of the data structure 49to produce an output structured electronic document, such as describedherein with regard to structured electronic document 800 (see FIG. 2).

The electronic device 11 can make the content elements 13 available to apresentation tier 42 for formatting for display as part of a userinterface screen 43 on a display 1104 of an electronic device 11, forexample, executing a computer program 39 on the processor 1100. Thecomputer program 39 can be for example a browser 40 with an extension 41sometimes referred to as a plug-in or add-in the browser 40. There aremany well known, commercially available software programs, commonlyreferred to as browsers, which can be executed on the electronic device11 as the browser 40. In FIG. 8 the document 15 is shown as accessibleon a remote server 47 through a network 48. The network 48 may be apublic network, such as the Internet, or a private network, such as anintranet. The remote server 47 is an originating server as thestructured electronic document 15 originates for the remote server 51based on CED 1 from the remote server 47. The computer programsdescribed herein, such as program 39, 40, 41, together with acomputer-readable medium, such as memory 1110, in which the program isstored can be a computer program product.

General techniques for generating an extension to a browser 40 arewell-known. Specific extensions 41 for use in association with a CED 1will be within the skill of those skilled in the art based upon suchgeneral techniques, their common general knowledge, and the informationcontained herein. In the case of a wireless device browser 40, limitssuch as screen size and bandwidth can be alleviated by displayingcontent rearranged or reduced in accordance with the CED 1 to the user.Rearrangement or reduction to take into account the limitations of aparticular device 11, 101 is often referred to as optimization.

In extracting content according to the CED 1 the browser 40 withextension 41 parses the CED 1 in order to identify the data elements 5,to determine the relationships between the data elements 5, to identifya respective content element 13 within a structured electronic document15 for each identified data element 5, and to extract the contentelements 13. The CED 1 can be structured such that publicly availableparsers can be used for this purpose. As is discussed herein, the CED 1can be structured in accordance with publicly available languages inorder to allow the extension 41 to use publicly available parsers wherepossible. The extension 41 can be provided with custom instructions forinterpreting the CED 1 for specific applications as required.

After retrieving the content elements 13, the browser 40 with extension41 can fill the data elements 5 with the extracted content elements 13.Referring again to FIG. 4, if desired, the data elements 5 can be storedin a data structure 49, including the entirety of the identified dataelements 5, 21 and the relationships between the data elements 5. Thedata structure 49 can then be made available to the presentation tier42. The content elements 13 are thus being made available to thepresentation tier 42 through the data structure 49. The extension 41 canutilize browser 40 features to carry out some functions of the extension41, for example, through application programming interface (API) callsfrom the extension 41 to the browser 40. The extension 41 is a contentextraction module similar to the content extraction module 53 discussedbelow.

The methods described herein are particularly well suited to wirelessnetworks. Accordingly, embodiments will be further described withreference to remote server 51 as network infrastructure of a wirelessservice provider in place of remote server 51. Although embodiments aredescribed herein with reference to the remote server 51 as networkinfrastructure it is to be recognized that other remote servers 51, notforming part of a network infrastructure could be used to carry out themethods described herein. It is also recognized that the term “remoteserver” is to be broadly interpreted and may, for example, includedistributed systems that distribute the functions of the remote server51 among more than one server. The remote server 51 will include one ormore processors, which can be similar to processor 1100, and associatedmemory, which can be similar to memory 1110, to store and execute thecomputer programs and instructions discussed herein.

Referring to FIG. 9, multiple CEDs 1 a, 1 b . . . 1 n can be stored foruse in association with multiple structured electronic documents 15 a,15 b . . . 15 n.

Referring to FIG. 10, in order to identify content elements 13 on astructured electronic document 15 for grouping, a user may select thecontent elements 13 on a user interface screen 200 where the structuredelectronic document 15 has been rendered to a display 1104. Selectionmay be performed, for example, by using a mouse or keyboard or otheruser input device 1102 (see FIG. 3). Example embodiments of techniquesfor content element 13 selection are further described with respect to atool embodiment later in this description.

A computer program running on electronic device 11, for example, anextension of a browser 40, contains computer-readable instructionsstored on a computer-readable medium accessible to the electronic device11 for execution on the electronic device 11 to associate the selectedcontent elements 13 with their respective content paths. Extensions arealso sometimes referred to as add-ins, add-ons, or plug-ins. Visualfeedback of user selection of a content element 13 can be provided, forexample, by highlighting the rendered content element 13 on the screen200, such as shown by the graphical outlines of rendered contentelements 13 a, 13 b, 13 c, 13 d, 13 g, 13 h, 13 i, 13 j. As shown inFIG. 10, FIG. 11 and FIG. 12, the graphical outline can include a cancelindicator 203, for example an encircled “x” the selection of whichdeselects the associate element of content 13.

Browser extensions 41 can have access to a DOM structure 63 (FIG. 7) ofa structured electronic document 15, and can utilize various browser 40features. For example, extensions 41 to a browser 40 distributed byMozilla under the trademark Firefox and available at www.mozilla.com canbe written using XUL (XML User Interface Language) and JavaScriptlanguages. Similarly, the browser 40 could be a browser distributed byMicrosoft Corporation under the trademark Internet Explorer. JavaScriptis a trademark of Sun Microsystems. Other browsers 40 may be used.

At the time of selection of a group, a user can input a name for thegroup. For example, the name can be input into a text box in a pop-upwindow in a similar manner to pattern naming described herein withrespect to FIG. 11. The name can then be incorporated by the computerprogram into the data structure definition 49 as the name of the grouparray element. This can simplify use of the data structure 49, asopposed to, for example, allowing the computer program to generate groupnames such as group1, group2, etc. Techniques for naming of element, forexample, groups, arrays, and other elements are discussed herein. It isto be recognized that other techniques for naming elements will beevident to those skilled in the art and can be used in place of thedescribed techniques.

In order to allow more efficient selection of elements for grouping, thecomputer program 41 can highlight on the user interface screen 200previously grouped content elements 13 for which content element 13grouping has previously been performed.

In order to identify groups, content element 13 paths can be in a formthat contains whole array elements. Methods of identifying contentelements 13 by paths are discussed herein. In addition, methods ofidentifying elements within content elements 13 paths that can berepresented as whole array elements are discussed herein. The computerprogram 41 can incorporate computer-readable instructions to carry outsuch methods to first express content element 13 paths in an arrayelement form with array element.

In order to allow more efficient selection of elements for grouping, thecomputer program 41 can highlight on the user interface screen 200content elements 13 paths that have been previously expressed in anarray element form as shown by the differently highlighted elements 13g, 13 h, 13 i, 13 j in FIG. 10, the highlighting being outlining in thisexample. Pattern recognition for Identification of content elements 13for array representation, and merger of array representation arediscussed below.

Referring to FIG. 12, individual rendered content elements 13 can benamed in a similar manner to groups using input text box 220 and pop-upwindow 222. These names can be used to name the data elements 5 of thedata structure 49.

Data structure definitions 4 for grouped data structures 49 can beutilized in, for example, a computer-readable content extractiondocument (CED) 1 for further use by a computer program executing on theelectronic device 11, such as browser 40 with extensions 41 to create aninstance of the data structure 49 in memory in accordance with the CED 1and fill the data elements 5 with corresponding content elements 13, andto utilize the filled data structure 49, for example, in a presentationtier 42 as discussed herein. It is recognized that the filled datastructure 49 could be used for other purposes, for example, automaticfilling of web forms, generation of word processing documents fromtemplates, or the population of a database for later use. Many otheruses will be evident to those skilled in the art based upon theprinciples described herein.

Grouping as discussed above allows grouping of separate data elements 5into complex types/structures 49. Grouping can improve the efficiencyand usability of the data structure 49.

Referring to FIG. 13, a method is shown to identify content of interestin a structured electronic document 15, such that, for example, thecontent can be used to form a common expression 3.

At 860, a structured electronic document 15 is rendered to a displaydevice 1104.

At 862, at least two separate indications of content elements 13 withinthe structured electronic document 15 are received through the inputdevice 1102.

At 864, an array of related content elements 13 within the renderedstructured electronic document 15 are identified with the processor1100. The array of related content elements 13 include the indicatedcontent elements 13 and a further content element 13 within thestructured electronic document 15. The further content element 13 isidentified as sharing a common characteristic with the indicated contentelements 13.

With reference to FIG. 14, a method of processing paths that identifycontent elements 13 within the structured electronic document 15 todefine a data structured definition 4, including relationshipdefinitions, according to a common expression 3 can involve at 150comparing a plurality of content paths, wherein each content pathidentifies a respective content element 13 in a structured electronicdocument 15. At 152, the method can further involve determining if thereis a plurality of content paths that have whole array elements anddetermine the least number of array elements in the paths. At 154, in adata structure definition 4 with data elements 5 corresponding to thecontent elements 13 identified by the content paths, and for each wholearray element in the determined content path, the method can furtherinclude defining an array group element in the data structure definition4 and nesting the corresponding data elements 5 under the array groupelement. A whole array element represents all elements of an array, forexample, T[X] is an entire array representing all individual elementsT[1], T[2], etc. of the array T where X is a variable index. Groupingcan bring together otherwise disconnected elements under a common parentcomplex element.

For example, three paths identifying content elements 13, such as aweather icon, text describing weather conditions, and text indicatingtemperature can be as follows:

/div#outer_wrapper/div#content_wrapper/div#stermfx/table/tr[0]/td[x]/img/src/div#outer_wrapper/div#content_wrapper/div#stermfx/table/tr[1]/td[x]/text/div#outer_wrapper/div#content_wrapper/div#stermfx/table/tr[2]/td[x]/textwhere td[x] is an array element, in this case an HTML table indicated tobe an array. A rendered example of a structured electronic document 15having the above paths might appear as elements 13 h, 13 i, 13 j, shownin FIG. 1 and discussed herein.

Without grouping as discussed in the method above, a data structure 49might define relationships between data elements 5 corresponding to thecontent elements 13 such that each data element 5 is a separate arrayelement 5 (for example: icon[x], conditions[x], temperature[x] extendingfrom a root element (responseElement) of the data structure 49), suchthat the data structure has data element paths:

responseElement.icon[x]

responseElement.conditions[x]

responseElement.temperature[x].

In this case “icon [x]” is an array of weather icons, “conditions [x]”is an array of weather conditions, and index for “temperature [x]” is anarray of temperatures.

As discussed previously a data element 5 is an element of the datastructure 49 whereas a content element 13 is a content element 13 of thestructured electronic document 15.

After grouping, a group array element (shortTermDay[x]) is inserted intothe data structure 49 and the whole array data elements 5 are convertedto individual data item elements 5 such that the individual dataelements 5 no longer represent whole arrays (e.g. icon [x] becomes icon)with the result that:

responseElement.shortTermDay[x].iconresponseElement.shortTermDay[x].conditionsresponseElement.shortTermDay[x].temperature

It is to be recognized that the grouping method can be performed on morecomplex content elements 13 paths such that group array elements may benested within group array elements.

For example, without grouping as discussed in the method above, a datastructure 49 might be defined as follows (with the initial contentelement 13 path on the left of the operator “=>” and the resulting datastructure 49 path on the right):

table/tr[x]/td[0]/text=> responseElement.bookTitle[x]table/tr[x]/td[1]/table/tr[y]/td[0]/text=>responseElement.label0[x].authorFirstName[y]table/tr[x]/td[1]/table/tr[y]/td[1]/text=>responseElement.label1[x].authorLastName[y]table/tr[x]/td[1]/table/tr[y]/td[2]/text=>responseElement.label2[x].authorRating[y].

After grouping, a second group array element (author[y]) is nestedwithin a first group array element (book[x]):

table/tr[x]/td[0]/text=> responseElement.book[x].bookTitletable/tr[x]/td[1]/table/tr[y]/td[0]/text=>responseElement.book[x].author[y].authorFirstNametable/tr[x]/td[1]/table/tr[y]/td[1]/text=>responseElement.book[x].author[y].authorLastNametable/tr[x]/td[1]/table/tr[y]/td[2]/text=>responseElement.book[x].author[y].authorRating.

Referring to FIG. 15, as a further embodiment of a method to create adata structure definition 4 defining a data structure 49, at 180, let Prepresent the set of content element paths selected for grouping. At182, let A represent the path with the smallest multiplicity (i.e.number of variable indices, for example “x”, “y” in the path shownabove). At 184, let P_(A) represent the paths in P with the samemultiplicity as A. At 186, let C be a root complex type element (e.g.responseElement) for data structure 49.

At 188, if P is empty, do nothing and exit at 189.

At 190, define a complex type G and the elements represented by thepaths in P_(A) are added as elements of complex type G.

At 192, add G as a field of C.

At 194, let C=G.

At 196, let P=P−P_(A).

Return to checking at 188 if P is empty and continue from there.

Referring to FIGS. 16 and 3, elements of paths for content elements 13that may be represented as array elements can be identified usingcomputer-readable instructions comprised in computer program 41 whichwhen executed on electronic device 11 carry out the following method.Typically the program 41 executing the method will be used by adeveloper of an output structured electronic document to define pathsthat identify content elements 13 in an existing structured electronicdocument 15, such that the content of the content element 13 can beextracted from the existing structured electronic document 15 in theoutput structured electronic document, such as page 800 (see FIG. 2);however, the program can be used to define paths to the content elements13 to use for other purposes.

To compare the paths of two content elements 13, at 102, starting fromthe path root, for each step in the respective paths, determine if theelement is the same. At 104, if any corresponding element in a step isnot the same, comparison ceases as a pattern has failed to berecognized. At 106, for corresponding steps where the respectiveelements are the same, identify if indices match in the two paths. At108, if so, the element is recognized at 110 one that can be representedas an array. Prior to pattern recognition at 101, at 100 structuredelectronic document 15 can be rendered to display device 1104 andpattern recognition utilizing arrays is a method of identifying with aprocesser 1100 a common characteristic between at least two contentelements 13 in a structured electronic document 15. At least twoseparate indicators of content elements within the structured elementdocument 15 can be received through input device 1106. At 114 furthercontent elements 13 in the structured document 15 that share the commoncharacteristic can be identified for example by iterating through arrayindices in a path. Further content elements 13 that share the commoncharacteristic can be highlighted of the display device 1104 to a user.

In either case, processing is returned at 112 to 102 if there are moresteps in the path.

When using the example path language described herein in further detail,a determination if the element is the same at 102 can be performed bydetermining if the element name and element ID are the same.

Where pattern recognition results in elements in the respective pathsthat can be represented as an array, the elements can be replaced in thepaths by an array element.

For example, given the two paths:

/div[4]/div[3]/table/tr[9]/td[2]/a/href/div[4]/div[3]/table/tr[10]/td[0]/a/href.

The resulting pattern will be identified as:

/div[4]/div[3]/table/tr[x]/td[y]/a/href.

Hence, n-dimensional arrays can be identified for paths according to theabove method, instructions and program 41. Such arrays can be identifiedfrom paths for two content elements 13. In the example above, a2-dimentional array is identified, and any element with the matchingpath (having indices in place of x and y) can be identified by the path.

In use, paths specified using n-dimensional arrays can be utilized byone or more computer programs 41 executing on electronic device 11 toidentify content elements 13 for example by iterating through the arrayindices based on a single path. This can provide a simple and efficientmethod to identify content elements 13 within a structured electronicdocument 15.

The paths can be used in, for example, a CED 1 for further use inidentifying content elements 13 in a structured electronic document 15.CED 1 can be used by one or more computer programs 39 to identifycontent elements 13. For example, a path can be used in a structuredelectronic document that is a CED 1.

Further aspects of pattern recognition including receipt of at least twoindications of content elements 13 within a rendered structuredelectronic document 15 are described herein with regard to an exampletool embodiment.

Content elements 13 within structured electronic document 15 can beidentified by paths that are similar to XPath paths, but with availablemarkup language elements to define the path selected from a subset ofmarkup language elements such that the path is less susceptible tochanges in the structured electronic document 15. For example, it ispossible to restrict the path to include only structural parents, i.e.the elements that define the structure of the document as opposed to itsdecoration. For example, in HTML, the <div> element defines structure,but the <b> element only defines element decoration. Ignoringnon-structural elements in a path improves element identificationimmunity to changes in the structured electronic document.

An example method to restrict the path to include only structuralelements for an HTML document 15 is to require the path nodes to beselected from a subset of HTML elements such as: head, div, span, form,table, thead, tfoot, tr, td, th, h1, h2, h3, h4, h5. In this case anyother elements not included in the subset would be considerednon-structural for the purpose of paths for elements within thestructured electronic document. It is recognized that the above listedsubset is not an exhaustive listing of all elements that might otherwisebe considered structural. Other structural elements could be included inthis subset if desired. For example, structured electronic documents inspecific content areas, for example documents using Chemical MarkupLanguage (CML), can have other markup language structural elements usedon a regular basis. The above listed subset is considered to result insufficiently unambiguous paths while resisting susceptibility to changesbetween structured electronic documents 15, for example between aninstance of a structured language document 15 used to create the commonexpression 3 and the data structure definition 4, and an instance of astructured electronic document 15 to which the common expression 3 anddata structure definition 4 are applied to extract content elements 13.

Paths can identify parent/child relationships, element indices, elementids, and the whole or partial array of elements.

An example of a path containing a reduced subset of HTML elements for anhref content element is: /div/div[4]/div#short/table/tr[x]/td/a/href,where:

‘/’ (forward slash) defines parent child relationship: parent/child,

‘[ ]’ (square brackets) defines element index for arrays:element[index], and

‘# short’ (number sign) defines element id:=“short” is merely an exampleid and not an HTML element.

A variable in place of an index indicates that the whole array isidentified (e.g. element[x]).

A polynomial in place of an index can indicate that a portion of thearray is identified (e.g. element[2x]—all elements with even index).

The last part of the path indicates what the path represents. If it isan attribute name, the path represents the value of that attribute; ifit is the keyword ‘text’, the path represents the textual content of theelement.

Further details of a possible path language for a subset of markuplanguage elements are described in Table 1.

TABLE 1 Path Syntax Description parent/child Indicates the element onthe right of the / is nested within the element on the left (i.e., theelement on the right is a child of the element on the left, which is theparent of the element on the right). parent/child#id Indicates the childof the specified parent with the specified id (the string to the rightof the #). parent/child[3] Indicates the fourth* element encountered asa child of the specified parent. The indices are increased by completingan inorder traversal of a document object model (DOM) tree whileignoring non-structural nodes when searching for children. Here ignoringessentially means treating the children of non-structural nodes aschildren of the non-structural node's parent node. *Counting begins at0. parent/child[x] Indicates all children of the specified parent.parent/child[2x+1] Indicates an array consisting of: child[1], child[3],child[5], etc. The elements within the array are determined by theexpression between the brackets (in this case, 2x + 1 - every oddindexed child). parent/child[x]/gchild[y] Indicates all grandchildren ofthe specified parent. A grandchild is a child of one of the parent'schildren. A common instance is table/tr[x]/td[y] to get all cells in atable.

For example, the path /div/div[4]/div#short/table/tr[x]/td/a/hrefidentifies an “href” element of content within the following HTMLstructure:

<html> ... <div> <div>...</div> <div>...</div> <div>...</div><div>...</div> <div> <div id=“short”> <table> <tr> <td> <ahref=“...”>...</a> </td> </tr> <tr> <td> <a href=“...”>...</a> </td></tr> ... <tr> <td> <a href=“...”>...</a> </td> </tr> </table> </div></div> </div> ... </html>.

It is understood that there could be any number of non-structural nodessurrounding the nodes specified in the path and that between the nodesspecified in the path, there could be any number of other nodes withdifferent names than those specified.

As the path does not include non-structural nodes, it is more resistantto subtle changes in the markup language code of the document than acorresponding path in accordance with the XPath.

Since the above path language captures only the structure of contentelements (by paths), it does not matter which search results pageinstance is used to choose content elements 13 for creation of a CED 1.

The structural element path language approach assumes that two pageinstances for the same page type have similar structure. For example,depending on which keyword is entered in an Amazon.com search, adifferent web page containing search results will be produced. Amazon isa trademark of Amazon.com, Inc. However, both search results pageinstances will correspond to the same structure. The structuralresemblance assumption will usually hold because dynamic web content isusually produced by the same template (e.g. a template scripted in ascripting language such as JSP, Active Server Pages (ASP), and PHP).

Also, the above path language allows an index variable to be used toindicate a list of elements. In the example above, this is shown on thetable row(tr) element, indicating that all table row (tr) elements ofthe parent table are to be included when the elements are fetched fromthe path.

While some content elements are independent; others can be combined intogroups. Grouping can be incorporated into the data structure 49 topermit a node array element that includes elements that are themselvesarrays. For example, a search of Amazon.com produces a structuredelectronic document 15 with a list of items. Each item has a title andprice. The data structure 49 can include a data element 5 that is anarray of complex types including title and price, rather than twoparallel arrays.

Element grouping can be used for better organizing content whendisplaying it to a user.

Referring to FIG. 17, the tool displays to the user a screen shot 548rendered version of the document 15 with content elements 13. Forexample, the document 15 can be rendered in a browser 40 window suchthat the document 15 will look similar to how the document 15 wouldordinarily appear in a browser 40 window.

In order to select content elements 13 in the document 15, the toolreceives a user indication from pointing device 1106 when a cursor 550is located over a content element 13 in the rendered document 15. Toassist in visual selection, the tool can highlight a content element 13at the pointing device before receiving an indication of the contentelement 13.

Selection ambiguity resolution can be utilized to identify a selectedcontent element 13 based upon a selected visible content element 13. Itis to be recognized, as with other aspects described herein, thatselection ambiguity together with or separate from visible selection canbe utilized separate from other aspects described herein to identify asselected content element 13. Selection ambiguity resolution cansignificantly assist users in working with structured language documentcontent elements.

It is to be understood that various aspects of the tool can be embodiedindependently of other aspects, or utilized in embodiments other thanthe specific tool embodiment described herein.

Referring to FIG. 18, for example, an embodiment of the tool will bedescribed that embodies a method to provide initial visual selection ofcontent elements at 400, selection ambiguity resolution 402, associate apath of content element 13 with data element path 403, array patternrecognition 404, array path conversion and array path merger 406, andarray grouping 408. Each of these aspects can be embodied independentlyof one another and of the tool, and each of these aspects may be usedindependently of one another.

Referring to FIG. 19, the tool stores gathered information in memory ina data structure 498, such as an object structure (for example, if thetool is a Firefox plug-in, the structure 498 would typically be based onJavaScript objects).

The tool keeps an OperationDefinition object 500 containing Elementobjects 502. Each Element object 502 points to two Path objects 504, onePath object 504 representing a data element path (elementPath 506)within data structure 49 and a second Path object 504 representing acontent element 13 path (htmlPath 508) in document 15. Each Path object504 contains PathPart objects 510 that make up the path. Each Elementobject 502 further specifies whether or not the object 502 represents anarray element in array property 512, and the type of the element in atype property 514, and a defaultValue property 516 for the element.

Referring to FIG. 20, an example data structure 498 contains sample datafor one OperationDefinition named Search with three Elements, Element1,Element2, Element3. Element1 has an elementPath, “iconURL”, and anhtmlPath, “/div[2]/img.src”. Element2 has an elementPath, “author[x]”,and an htmlPath, “/div[2]/table/tr[x]/td/text”. Element3 has anelementPath, “temperature”, and an htmlPath,“/div[0]/table/tr[3]/td/text”. Element1 and Element3 are not arrays,while Element2 is an array.

Although the data structure 498 is described for one operation, theprinciples can be extended to multiple operations in which case therewill be multiple OperationDefinition objects in the data structure inmemory.

Referring to FIG. 21, many HTML nodes are invisible but still clickableon the rendered page. For example, a hyperlink content element 13 is notvisible by itself; it covers other visible content elements 13 such asimage or text. Where one element of content 13 surrounds another elementof content 13, the elements of content 13 are coincident In order toresolve ambiguities between visible and invisible clickable elements,the tool at 450 by default initially recognizes the visible contentelements 13 as the selected content elements 13. It is recognized thatthe tool could default to the invisible element if desired. The tool canreceive instructions regarding the content element 13 being selected bythe user to assist in resolving the ambiguity, such as for examplewhether the content element 13 is text, hyperlink, or image.

Referring to FIGS. 17 and 21, for example, the tool can determine at 452if there is an ambiguity with respect to a type of the content element13 when the cursor 550 is positioned over the location of a renderedcontent element 13, and, if so, at 454 the tool can provide the userwith the ability to access a right click menu 552 from which one of thepossible content element 13 types may be selected. Once the type isselected, the tool at 456 seeks an associated content element 13 thatmeets the selected type. If at 457 an associated content element 13 isfound then the tool at 458 recognizes a found content element 13 as theselected content element 13. If the tool cannot find an associatedcontent element 13 then the tool fails to find an associated contentelement 13 and returns an error message at 460 to the user, for examplethrough the display 1104. For example, the error message could be a“selected content element type not found”.

In another example, non-content elements, for example structuralelements such as, for example in HTML, tables “<table>”/divs“<div>”/rows “<tr>”/columns “<col>”/etc surrounding the desired contentelement 13 may not be visible, but they may be clicked if a cursor 550is outside a clickable portion of a desired visible content element 13when a user click is received content element. For example, a user maybe attempting to select a text content element 13, but the cursor 550 islocated outside the text content element 13, over a table row (tr)element containing the desired text content element 13.

Referring to FIG. 22, the tool assumes that a user is attempting toselect content elements 13, such as for example text, hyperlink, orimage. The tool at 470 examines the initially selected element todetermine if it is one of the recognized types for a content element 13.If so, then the tool has resolved a non-content type ambiguity and thecontent element 13 is recognized at 472 as the selected content element13. If not, then the tool at 474 seeks a recognized type of contentelement 13 that is associated with the selected element. If the tool at476 finds a recognized type of content element 13 then, unless there isa further ambiguity at 478 in a content element 13 that is found, thetool at 480 recognizes a found content element 13 as the selectedcontent element 13. If the tool cannot find an associated contentelement 13 at 476 then the tool fails to find an associated contentelement 13 and returns an error message at 482 to the user, for examplethrough the display 1104. For example, the error message could be“Associated content element not found.”. At 478 and 484, the tool can,for example, utilize the prior method beginning at 452 to resolve acontent type ambiguity before the tool at 480 recognizes a found contentelement 13 as the selected content element 13.

Referring to FIG. 23, as a further alternative example the tool can beprogrammed to execute on the electronic device 11 a computer programcontaining instructions to the computer to execute in accordance withthe following method to select a desired content element 13 based uponan element identified through a user input device from a renderedstructured electronic document 15.

An example embodiment of the method including pre-order traversalthrough the identified element's children can perform as follows:

-   -   490/492: If at 492 a content element 13 is found in the        identified element then at 494 stop and recognize the found        content element 13 as the selected content element.    -   490/494/496: If at 490 a content element 13 is not found and at        492 the identified element is a root element then at 496 return        an error message to the user, for example through display 1104        that a content element 13 cannot be found for the identified        element, so that a user can make a further attempt to click on a        desired content element 13.    -   490/494/498: If at 490 a content element 13 was not found and at        494 the identified element is not a root element then the tool        performs repeats at 490 for each of the identified element's        parents as the identified element until the document 15 root        element is reached and an error is recognized at 496, or a        content element 13 is found and recognized at 492.

Referring to FIG. 24, as an example, starting from a “Hello” textcontent element 13 m identified through a user input device 202 from arendered structured electronic document 15. identified element Hellotext 13 m, the tool will try to find an associated hyperlink element 13n. Example HTML might be:

<div>Text1</div> <div><a href=”/abc.html”>Hello</a></div><div>Text2</div>.

The DOM for the structured electronic document is represented in a DOMtree 600 in FIG. 24.

The user indicates desire for a hyperlink content element 13 (forexample using a right click menu as discussed previously) and clickswith the cursor 550 over the “Hello” text element 13 m. The tool checkschildren first (no children), and then it will find the hyperlinkcontent element 13 n as clicked element's (“Hello” text13 m) parentelement.

Referring to FIG. 25, as a further example, given the HTML:

<div>Text1</div><div><table><tr><td>Hello</td></tr><tr><td>Text2</td></tr></table></div> <div>Text3</div>a user would like to select a text content element 13 p (desire for atext element is indicated by default), but clicked on a non-contentelement 602, a table cell (the cursor was outside a clickable portion ofthe text element of contact 13 p) and because the table was invisible,the user did not know he clicked on a table row element 602. The toolchecks children first and finds the text content element 13 p.

The tool also determines a markup language path of a selected elementfor example content element 13 p from the example described immediatelyabove. A markup language path can be determined, for example, byperforming a pre-order transversal through the elements of document 15starting from the root and looking for the selected element 13 p. Whiledoing traversal the tool can record element indices for elements in thepath.

An example Java-based pseudo-code embodiment of the method can beexpressed as follows, where elements of the document 15 are referred toas nodes:

Path selectedNodePath = getPath(emptyPath, documentRootNode,selectedNode); public Path getPath(Path path, Node root, NodeselectedNode) { if(node != null) { Map<String, int> indices = newMap<String, int>( ); for(Node child : node.getChildren( )) { StringchildName = node.getName( ) int index = indices.get(childName) + 1indices.set(childName, index) Path childPath = path + newPathPart(node.getName( ), index) if(child == selectedNode) returnchildPath; else { Path selectedNodePath = getPath(childPath, childNode,selectedNode) if(selectedNodePath != null) return selectedNodePath } } }}

As previously described with respect to FIG. 12, the tool shows a pop-upwindow 222 asking the user to enter the element name for the selectedcontent element 13.

The user enters an element name and clicks button 224 ok. Additionaldata input fields such as input boxes or selection buttons, not shown,can be provided on pop-up windows 222, or further pop-up windows, notshown, can be provided, for the user to specify element type such asstring, integer, date, duration, etc. Element type information canenable content extraction engine 61 to produce a more useful DOMstructure 49. For example, if an element of type date is specified,content extraction engine 61 puts a date object (as opposed to string)in data structure 49, for example by loading data in ISO 8601 format.This would allow presentation tier 42 to format date as specified in thetemplate. Similarly, further date entry fields or pop-up windows, notshown, can be provided for the user to specify an element default valueand the content extraction engine 61 would substitute a default value inthe data structure 49 if the value is not found in the extractedcontent.

Following element naming as set out above the tool adds a new element 5for the selected content element 13 to the data structure 49 as follows,assuming that the markup language of the document 15 is HTML:

-   -   a. Element path=<elementName>. E.g. temperature, as named by        user above    -   b. HTML path=<path determined as described above>    -   c. Array=false    -   d. Type=<set if entered by user>    -   e. Default value=<set if entered by user>

Referring again to FIG. 17, following element naming and addition to thedata structure 49 the tool highlights the selected content element 13 inthe rendered structured electronic document 15. The selected contentelement 13 can be highlighted for example by providing a colored overlayover the selected content element 13 as represented by dashed outline606. Other highlighting techniques, such as for example an outline, willbe evident to those skilled in the art.

At this point the tool has completed user selection of a content element13 from the structured electronic document 15. The path of the contentelement 13 in the structured electronic document 15 is known, named andstored.

Referring to FIG. 26 and the previous description with respect to FIG.11, an embodiment of an example pattern recognition method incorporatedinto a tool can involve a user selecting at least two elements 13.Typically a user will select elements that the user believes participatein a desired pattern. Element selection can be performed, for example,as described above by indication through a pointing device 1106. Forpattern recognition, the tool, for example, keeps a Pattern object 700containing Element object 702 for content elements 13 to be consideredin pattern recognition (sample elements) and resulting array elementinformation.

A pattern can be determined by selecting two elements participating inthe pattern. However, to potentially improve refinement of patternrecognition, the tool can allow choosing more than two elements forpattern recognition.

For each selected content element 13, a user holds a cursor over theelement 13 and provides a key press, such as pressing a plus sign, “+”key on a keyboard 1108 to indicate the selection of the element 13 forpattern recognition. Other key presses or indications could be used toindicate the selection of the element for pattern recognition. The “+”key press is received by the tool as a user indication that patternrecognition is desired for the selected element 13. Once the “+” key ispressed, the element 13 is highlighted to indicate that it is to be partof pattern recognition, and a corresponding Element object 702 for theselected element 13 is added to the Pattern object 700 as a sampleelement. A previously indicated content element 13 can be cancelled, forexample, by receiving from the pointing device 1106 an indication tocancel through a cancel indicator 1120 such as cancel indicator 1120.

Once two elements 13 have been chosen in the pattern, the tool shows apop-up window 212 asking the user to enter a pattern name for thepattern in a textbox 210 (see FIG. 11). The user enters the array name(optionally the user can enter element type and default value) andindicates acceptance of the user name for example by indicating ““OK” onbutton 214.

With the selected elements the tool performs pattern recognition on themarkup language paths (HTML paths in the examples used herein, althoughpattern recognition is not limited to HTML paths) associated with theElements in Pattern object (elements selected for pattern recognition).Pattern recognition outputs a single markup language path for theselected elements containing variables in place of some indices (anarray element path).

The tool can, for example, perform array pattern recognition and arraypath conversion in accordance with the following pseudocode:

Input: Set of HTML paths

Output: A single HTML path containing variables or error

1. Let P be the set of HTML paths

-   -   Let t be the resulting HTML path

2. Let t=the first path from P

3. For each path p in P do:

-   -   a. If p and t have a different number of parts, fail pattern        recognition and exit    -   b. Let L=length of t    -   c. For j=0 to L−1 do:        -   i. If j-th part name in p is different from j-th part name            in t, fail pattern recognition and exit        -   ii. If j-th part index in p is different from j-th part name            in t, array pattern is recognized and the path is converted            to an array path by replacing j-th part in t with a variable            (x,y,z, . . . ). Variable names should be unique within path            only.

4. return t as the result

The tool can then create a corresponding array data element path(Element Path) for the array content element path (htmlPath) createdabove. For example, the tool can determine an array data element pathemploying an embodiment of a method reflected in the followingpseudocode:

-   -   a. Path arrayElementPath=new Path( );    -   b. For each variable in array htmlPath path going from left to        right do:        -   i. Add a new path part to arrayElementPath the path part            name is globally unique artificial path part name like L0,            L1, . . . and the array index is the variable.    -   c. Add a new path part to arrayElementPath where path part name        is a user-entered array name.

For example, given an HTML path table/tr[x]/td[y]/text[x], and userentered array name “author”, the array data element will beL14[x].L15[y].L16[z].author.

The tool then creates an Element object and sets this element asarrayElement in the Pattern object. The created Element object contains:

a. Element path=arrayElementPath (from above)

b. HTML path=array htmlPath from above

c. Array=true

d. Type=<set if entered>

e. Default value=always NULL

For each sample element in the Pattern object, the element is removedfrom the OperationDefinition object 500. An arrayElement from thePattern object is added to the OperationDefinition object 500. Thisresults in a merged array content element 13 path for the selectedelements. This also generates a data structure 49 definition withcorresponding merged array data elements 5.

The tool then highlights all elements whose HTML element paths match thearray HTML element path (e.g. by providing a colored overlay over theselected node as discussed previously) from arrayElement in Patternobject. For example, the literal paths “div/table[0]/tr[0]/td/text” and“divitable[0]/tr[1]/td/text” each match array element path“div/table[0]/tr[x]/td/text path” and would be highlighted as a match.

The tool has then completed array pattern recognition, array pathconversion, and array path merger.

The tool can further incorporate grouping of array data content elements13. For example, the tool may embody instructions to carry out thefollowing method:

-   -   1. The tool receives user selection of two or more elements 13        from different array elements 13. Element 13 selection can be        performed in a similar manner to selection of elements 13 as        previously described.    -   2. For each array element the user wants to add to the group,        user holds mouse over any element of the array and provides a        key press, for example a key press of “+” (or some other        indicator to the tool of user desire to add array to the group)        to add the array to the group as discussed previously for        indicators for pattern recognition. Once “+” is pressed, the        array element 13 is highlighted to indicate that it has been        selected to participate in the group, and the corresponding        Element object is added to a Group object, in a similar manner        that Element Object 702 is added to a Pattern Object 700.    -   3. Once one element 13 has been added to the group as described        above, the tool shows a pop-up window asking user to enter group        name (as described previously).    -   4. User enters group name (or multiple names, e.g.        dot-separated) and clicks ok.    -   5. Group data elements in data element path, for example, in        accordance with the following pseudo-code:        -   a. Let n=length (number of path parts) of the longest            element path in elements in Group object.        -   b. Let groupPartName be an array of length n−1 field with            globally unique group part names (G0, G1, etc). If group            names were specified, then group names will be used instead            of G0, G1, G2. For example, if book.author was specified            than groupPartName={book, author};        -   c. For each element in Group object do:            -   i. for(int j=0; j<elementPath.length−2; j++) do:                -   1. elementPath.part[j].name=groupPartName[j]    -   6. End.

For example, if elements with the following data element paths areselected for grouping:

L22[x].L23[y].name L31[x].L32[y].age L17[x].price L15[x].titlethen, in accordance with the above, the resulting data element pathswill be:

G7[x].G8[y].name G7[x].G8[y].age G7[x].price G7[x].title

If “book” and “author” are specified as group names, the resulting dataelement paths will be:

book[x].author[y].name book[x].author[y].age book[x].pricebook[x].title.

The tool can then generate a content definition, which may for exampleform a CED 1. The content definition has the common expression 3 anddata structure definition 4 previously referenced. The data structuredefinition 4 defines the structure for instances of data structure 49.The service definition, service input definition, request message,response message, and port type, and bindings all form part of a commonexpression 3 identifying content elements 13 in a structured electronicdocument 15. The data structured definition 4 is defined according tothe common expression as set out above. For this example, it is assumedthat the operation name is “Search”.

Content definition generation is initiated, for example, by the toolpresenting a user input interface to a user to request generation. Suchan input interface might be, for example, a “generate WSDL” button on atoolbar. Other forms of input interfaces, such as for example a menuitem, will be evident to those skilled in the art. As an alternative,the content definition may be generated automatically as requiredinformation is obtained by the tool.

As part of content definition generation the tool, for example, forcontent definition in a WSDL format as discussed elsewhere herein, awsdl: service can contain a wsdl:port further containing an html:addresslocation element pointing to an endpoint URL. For example:

<wsdl:service name=“Service”> <wsdl:port name=“Port”binding=“impl:Binding”> <html:address location=“http://[Hostname]/service”/> </wsdl:port>

The tool then generates primitive string elements for each inputparameter under requestElement complex type for types section. Forexample:

<xs:element name=”requestElement”> <xs:complexElement> <xs:sequence><xs:element name=″keyword″ type=″xs:string″/> <xs:elementname=″category″ type=″xs:string″/> </xs:sequence> </xs:complexElement></xs:element>

The tool then generates data structure definition 4 elementscorresponding to element paths in OperationDefinition underresponseElement complex type for types section. For example, elements 13with paths:

query book[x].author[y].name book[x].author[y].age book[x].pricebook[x].titlewill result in a data structure definition 4 of a data structure 49:

<xs:element name=”responseElement”> <xs:complexElement> <xs:sequence><xs:element name=″query ″ type=″xs:string″/> <xs:element name=”book”maxOccurs=”unbounded”> <xs:complexElement> <xs:sequence> <xs:elementname=”author” maxOccurs=”unbounded”> <xs:complexElement> <xs:sequence><xs:element name=″name″ type=″xs:string″/> <xs:element name=″age″type=″xs:integer″/> </xs:sequence> </xs:complexElement> </xs:element><xs:element name=″price″ type=″xs:double″/> <xs:element name=″title″type=″xs:string″/> </xs:sequence> </xs:complexElement> </xs:element></xs:sequence> </xs:complexElement> </xs:element>.The service output from the service output definition is the datastructure 49.

If element type and default values are specified, they can be includedin the data structure definition 4 as part of the element definition,for example a type of “xs.string” is provided for the title elementabove.

The tool then generates a request message, for example:

<wsdl:message name=“SearchRequest”> <wsdl:part name=“requestElement”element=“impl:requestElement”/> </wsdl:message>.

The tool then generates a response message, for example:

<wsdl:message name=“SearchResponse”> <wsdl:part name=“responseElement”element=“impl:responseElement”/> </wsdl:message>.

The tool then generates a portType:

<wsdl:portType name=“SearchInterface”> <wsdl:operation name=“Search”><wsdl:input message=“impl:SearchRequest”/> <wsdl:outputmessage=“impl:SearchResponse”/> </wsdl:operation> </wsdl:portType>.

Tool then generates a binding for each element in OperationDefinitionobject including htmlPath and bindingPath (elementPath), for example:

<wsdl:binding name=“Binding” type=“impl:SearchInterface”><wsdlsoap:binding style=“document”transport=“http://schemas.xmlsoap.org/soap/http”/> <wsdl:operationname=“Search”> <wsdl:input name=“SearchRequest”> <wsdlsoap:bodyuse=“literal”/> </wsdl:input> <wsdl:output name=“SearchResponse”><wsdlsoap:body use=“literal”/> <html:htmlBinding> <html:nodeBindinghtmlPath=“/div#outer_wrapper/div#content_wrapper/text[2]”bindingPath=“responseElement.query”/> <html:nodeBindinghtmlPath=“/div#outer_wrapper/table/tr[x]/td[5]/table/tr[y]/td[2]/text ”bindingPath=“responseElement.book[x].author[y].age”/> <html:nodeBindinghtmlPath=“/div#outer_wrapper/table/tr[x]/td[5]/table/tr[y]/td[3]/text ”bindingPath=“responseElement.book[x].author[y].name”/> <html:nodeBindinghtmlPath=“/div#outer_wrapper/table/tr[x]/td[0]/text” bindingPath=“responseElement.book[x].title”/> <html:nodeBindinghtmlPath=“/div#outer_wrapper/table/tr[x]/td[4]/text”bindingPath=“responseElement.book[x].price”/> </html:htmlBinding></wsdl:output> </wsdl:operation> </wsdl:binding>.

The CED 1 can be associated with one or more presentation templates 57,such that the content elements 13 retrieved from the document 15 can beput in a format for display by the electronic device 11 or the device201. The templates 57 can be registered with the remote server 51 in amanner similar to the CED 1. Alternatively, if a presentation template57 is not available, the content elements 13 can be sent as datastructure 49, for example in JavaScript Object Notation (JSON) or XMLformat, to electronic device 11 or device 201.

It is to be recognized that a CED 1 can be associated with apresentation template 57 such that content extraction according to CED 1can be used to initiate presentation generation according to template57. For example, once the content elements 13 have been extracted thenthe CED 1 invokes a presentation tier 42 to perform presentationgeneration according to template 57. A CED 1 and presentation template57 can be otherwise associated with one another. For example, thepresentation template 57 can trigger content extraction according to CED1, rather than pre-fetching content according to CED 1 and theninitiating presentation generation.

A series of CEDs 1 and presentation templates 57 may be strung togetherto perform a sequence of operations on structured electronic documents;each CED 1 or template 57 calling the next, or being called under thecontrol of a main CED 1 or template 57. For example, it may be requiredto enter information at a search request page prior to receiving searchresults. A CED 1 can be used for the search request page with apresentation template 57 to display the search request page. A furtherCED 1 can be used for the search result page. For interactions requiringmultiple CEDs 1 without user input, multiple CEDs 1 can be initiatedsequentially before a presentation template 57 provides information fortransport to a user. Multiple operations can also be defined in a singleCED1, for example using multiple port types in a WSDL format. Otherconfigurations of CED 1 and presentation templates 57 can be possibledepending on the allowed methods of retrieving the document 15. Forexample, it may be possible to directly address the search result pagein an URL to prompt the return of a results page in one step. An examplepresentation template 57 for the weather example discussed herein mightinclude the following template, where {@elementPath} is replaced with avalue for the specified element path for the data elements 5 in the datastructure 49, and {*arrayPath}html{*arrayPath} repeats the specifiedHTML for each array element in the array specified by arrayPath:

<html><body> Weather for: <b>{@responseElement.location}</b><br/><br/><br/> <table> <tr> <td> <imgsrc=“{@responseElement.icon}”/><br/><br/> </td> <td>{@responseElement.conditions} {@responseElement.temperature}{@responseElement.temperatureUnits} {@responseElement.rain}{@responseElement.snow} </td> </table> <br/> <table>{*responseElement.shortTermDay} <td> {@responseElement.shortTermDay.day}{@responseElement.shortTermDay.time} </td> <td> <img src=“{@responseElement.shortTermDay.icon}”/> </td> <td width=“20”>&nbsp;</td><td> {@responseElement.shortTermDay.conditions}{@responseElement.shortTermDay.temperature} </td> </tr> {*responseElement.shortTermDay}</table> </body></html>

Such a template 57 would render an output structured electronic document800 similar to that shown in FIG. 2 for the weather example describedherein. One can see that the page 800 can be more compact and mobilebrowser screen friendly. It can also exclude elements of the structuredelectronic document 15 that might not be renderable by a mobile browser.Other formats of the output structured electronic document 800 arepossible at the discretion of the designer. The benefits of the outputstructured electronic document 800 are not limited to presentation onmobile devices, and can extend to other platforms.

Referring again to FIG. 7, on a personal content delivery embodiment auser registers a content extraction document CED 1 with a remote server51, such as a computer executing a computer program and accessiblethrough a public or private network 48 a. The CED 1 defines a commonexpression for the remote server 51 to find a respective content element13 within a structured electronic document 15. The structured electronicdocument 15 can, for example be located on a remote server 47 accessiblethrough a public or private network 48 b. The remote server 51associates the CED 1 with the user. The remote server 51 can use avariety of computer-based techniques to associate the CED 1 with theuser. For example, the remote server 51 can associate the CED 1 with auser account that requires authentication, such as a password, from theuser. The remote server 51 can associate the CED 1 with an address, forexample an Uniform Resource Identifier (URI), on remote server 51 thatis known to the user. The remote server 51 can associate the CED 1 witha particular electronic device 11. The electronic device can beidentified when the electronic device 11 communicates with the remoteserver 51 through the network 48 a. The electronic device 11 can be, forexample a wireless handheld device 201, such as for example by a deviceserial number transmitted from the electronic device 11 to the remoteserver 51.

As the device 201 is an electronic device 11, it contains similarcomponents, although in alternate configurations appropriate to ahandheld format. Accordingly, similar reference numerals for thecomponents of the device 11 will be used for the components of thedevice 201, for example, display device 1104, input device 1102,processor 1100, keyboard 1108, pointing device 1106, and memory 1110.For simplicity, the description of the components of the device 201 willnot be repeated, nor will the components of the device 201 beindependently identified in the FIGS.

The remote server 51 can utilize cookies that identify the electronicdevice 11 associated with the user. A combination of techniques toassociate a user with the CED 1 can be used. Such techniques can be usedsequentially, such that each step must be performed, or in parallel toallow alternative ways to associate the CED 1 with a user.

Data to determine the association of the CED 1 with the user whenextracting content according to the CED 1 is stored on a computerreadable medium, which is accessible to the remote server, such asmemory or a disk drive.

Content element structured electronic document. The CED 1 can be createdat an electronic device 11 executing a computer program 39. The computerprogram 39 can, for example, comprise a browser 40 with extension orplug-in 41 as discussed herein. It is recognized that the electronicdevice 11 can be a wireless device, though it is further recognized thata handheld wireless device can be limited in its speed and screen sizeamong other resources, which can make it impractical for this purpose.Alternatively, other computer programs running on a computer can beprovided for use in creating a CED 1. Computer program 39 for creating aCED 1 could be accessed remotely. For example, the wireless serviceprovider could run a web-enabled application on a remote server forusers to create a CED 1. For ease of use, a CED 1 may be createdutilizing a desktop computer as the electronic device 11 with a display1104 (see FIG. 8) of sufficient size for the work, and to avoid anywireless network charges.

Automated generation of a CED 1 can be implemented through a tool suchas the tool described herein. Such automated generation can allow a CED1 to be created without access to underlying coding technologies, forexample structured definition languages such as HTML, XML, or WSDL. Thismakes it possible for users to create their own CEDs 1 for registration.It is also possible for developers to generate CEDs 1 in an automatedmanner. Although developers can have the knowledge necessary to manuallycreate a CED 1 based upon the principles described herein, the use of atool with automated generation capabilities can make the creation muchmore efficient.

Upon registration, the remote server 51 provisions the CED 1 on thenetwork, such that the remote server 51 will initiate content extractionaccording to the CED 1 when the remote server 51 receives a request toprocess content according to the CED 1 associated with the user. As anexample, the identifier could be a uniform resource locator (URL)pointing to a location in the remote server 51 that the remote server 51associates with the CED 1. The identifier can also be the identifier forfinding the original structured electronic document 15 (such as the URLof the document 15), and the remote server 51 can be provisioned toinitiate content extraction according to the CED 1 whenever the useruses the original URL. As a further alternative, if a remote serverother than the remote server 51 is used, then the identifier coulddirect the electronic device 11 to the remote server. The identifiercould, for example, be saved as a favorite by the user on the electronicdevice 11.

When the user wishes to view the content elements 13 from the structuredelectronic document 15 as defined in the CED 1, the user uses theidentifier to initiate content extraction according to the CED 1 by theremote server 51. The remote server 51 accesses the CED 1, finds thecontent elements 13, and transmits them to the electronic device 11 in aform suitable for rendering to a display 1104 (see FIG. 8) for the user.Rendering may occur in a browser 40.

It is to be recognized that the computer for creating the CED 1, theelectronic device for registering the CED 1 with the remote server 51,and the electronic device for viewing the content elements 13 need notbe the same electronic device. (Each of these electronic devices isexemplified by the electronic device 11.) For example, the computer forcreating the CED 1 can be a desktop computer. For example, the computerfor registering the CED 1 could be either a desktop computer or awireless handheld device. Requests to initiate content extraction can bemade from the computer at which the user wishes to view the contentelements 13. The remote server 51 can be set to recognize requests fromthe computer as a request from the user, provided the computer meets anyauthentication requirements, such as those discussed below.

The remote server 51 may be restricted to recognize requests from aparticular electronic device 11 associated with the user's account, forexample, a wireless handheld device 11. Similarly, transmissions in aform suitable for rendering to a display 1104 could be restricted to aparticular electronic device 11 associated with the user's account, forexample, a wireless handheld device 201. For example, in the case of awireless handheld device 201, device restrictions can be enforced basedon a device identification number associated with a user account. Apassword, cookie or other authentication technique could be used where,for example, a device serial number is not available.

The CED 1 could be registered in many ways. If using a wireless handhelddevice 201, the CED 1 could be downloaded to the wireless handhelddevice 201 for uploading to the remote server 51. Alternatively, the CED1 could remain on the remote server 51 associated with a user's accountif created there by a user. Then a user could use a wireless handhelddevice 11 to register the CED 1 with the remote server 51, for example,by using a web interface or sending an email to the infrastructure 51.

After successful provisioning on the remote server 51 of CED 1, once theelectronic device 11 or device 201 makes a request to initiate contentprocessing according to CED 1, the remote server 51 obtains the contentelements 13 defined in the CED 1 and transmits them to the electronicdevice 11 or the device 201 in a form suitable for rendering to adisplay 1104 of the user.

The computer readable storage medium can be, for example, a randomaccess memory, storage area network (SAN), magnetic disk drive (harddrive) or other computer-readable storage medium accessible to theremote server 51.

Further example details for creating and for using an example CED 1 forpersonalized content delivery have been previously set out. Thesedetails are being provided as examples only. Within the exampledescription have been described other aspects and embodiments that haveapplicability beyond the creation and use of a CED 1. Within the exampledescription have been described other aspects and embodiments that haveapplicability to personalized content delivery. It is to be understoodthat such other aspects and embodiments are included within the scope ofthe principles described herein, and not limited to the creation and useof a CED 1, or to personalized web delivery. Also, personalized webdelivery could be carried out utilizing other forms of CED 1 created inanother manner, or utilizing the techniques described herein other thanwith a CED 1.

In this description various aspects and embodiments of such aspects,have been described. For example, in an embodiment of an aspect there isa method of identifying content of interest in a structured electronicdocument by an electronic device having a processor, an input device,and a display device. The method can include rendering a structuredelectronic document to the display device; receiving through the inputdevice at least two separate indications of content elements within therendered structured electronic document; and identifying with theprocessor a common characteristic of the indicated content elements, andidentifying any further content element within the rendered structuredelectronic document sharing the common characteristic with the indicatedcontent elements.

Prior to rendering, the structured electronic document can be obtainedfrom an originating web server. The method can further include,following receipt of the at least two separate indications, individuallyhighlighting in the rendered structured electronic document each of theat least two separately indicated content elements.

The input device can include a pointing device and, receiving the atleast two separate indications of content elements can include receivingeach indication from the pointing device when applied at a displayedlocation of a content element in the rendered structured electronicdocument.

The method can include displaying a cancel indicator adjacent apreviously indicated content element for receiving from the pointingdevice an indication to cancel the previously indicated content element.The method can include highlighting a content element when the pointingdevice is located at the control element prior to receiving anindication of the content element.

Receiving at least two separate indications of content elements caninclude identifying a content element surrounding an indicatedstructural element. Receiving at least two separate indications ofcontent elements can include offering a choice to indicate a contentelement from a plurality of coincident content elements.

The input device can include a keyboard and, receiving at least twoseparate indications of content elements can include receiving a keypress at the keyboard.

The method can include displaying a text box on the display device forreceiving input of a name for a content element, and storing theinputted name.

In another example aspect an embodiment can include a computer programproduct comprising a computer program stored on a computer-readablemedium. The computer program product can include instructions which whenexecuted by an electronic device having a processor, an input device,and a display device cause the electronic device to carry the method ofthe above aspect.

The instructions can be extensions of a browser forming part of theelectronic device for rendering structured electronic documents to thedisplay device. The extensions can have access to a DOM tree of thestructured electronic document, and the extensions can utilize variousbrowser features.

In a further example aspect an embodiment provides an electronic deviceincluding a stored computer program, a processor executing in accordancewith the stored computer program, an input device, and a display device.The computer program includes instructions which when executed by theelectronic device to carry out the method of an aspect above. Theinstructions can be extensions to a browser.

Other aspects and embodiments are also described in the abovedescription, and other aspects and embodiments will be evident from theabove description.

Throughout this description various components are described along withmethods and apparatuses for utilizing such components. Unless thecontext requires otherwise, such methods are carried out utilizingcomputer-readable instructions stored on a computer-readable mediumaccessible to a computer to cause the computer to carry out the methods.Such computer-readable instructions to carry out the methods form partof the description herein.

This description describes many different embodiments of a variety ofaspects. Among other reasons, some embodiments are being describedtogether to provide example context for ease of understanding. Althoughsome aspects may be incorporated within or together with other aspects,it is to be understood that the scope of the embodiments and aspectsdescribed herein is not limited to incorporation within or together withsuch other aspects.

As use of structured electronic documents is best known for rendering inbrowsers, this description typically refers to examples utilizingbrowsers; however, it is to be understood that the embodiments andaspects are not limited to structured electronic documents that arerendered only in browsers. For example, email clients and otherapplications can also render structured electronic documents throughembedded rendering engines or API calls to a rendering engine.

As an example, some embodiments relate to a content extraction document(CED) 1, examples of which are discussed herein with particularreference to FIG. 3. Other embodiments relate to systems and methodsthat perform content extraction according to a CED 1, and still furtherembodiments relate to methods and tools for creating a CED 1. Furtherembodiments relate to methods and system that utilize a commonexpression to provide content elements 13 from a structured electronicdocument 15. Other example embodiments include methods and systems,independently or in combination, for identifying content elements 13,for selecting content elements 13, for converting elements in a path ofa content element 13 to arrays, and for grouping elements in a path of adata structure associated with array elements in paths of contentelements 13.

Further, it is understood that features and functions of one embodimentor aspect may be utilized in other embodiments and aspects withoutrequiring that the description of the features and functions for the oneembodiment or aspect be repeated in this description for the otherembodiments and aspects.

One of skill in the art will appreciate that the methods describedherein can be used with the apparatuses described herein, but could alsobe used with other apparatuses. Accordingly, some primary steps arepresented in a generalized form that does not rely on the particularapparatuses described herein. It is noted in the description of certainsteps and substeps that such steps can be performed by specific elementsof the apparatuses; however, the association of steps and apparatuses isdone by way of example but not limitation, and it is to be understoodthat these steps could be performed by other apparatuses. Moreover, theterm “step” is used to refer to both the general steps associated withthe methods and to more detailed substeps which can be comprised as partof a more general step. Some steps are optional. Optional substeps canbe omitted or replaced by other specific method steps that implement orembody the function of the primary step. Although discrete steps arementioned, it will be understood by one of skill in the art that in someembodiments, the functions defined in the steps can be performed ascontinuous processes.

The steps or operations described herein are just for example. There canbe many variations to these steps or operations. For instance, the stepscan be performed in a differing order, or steps can be added, deleted,or modified. Parts of one embodiment can be used in another embodimentwithout requiring all of the steps of any one embodiment.

The embodiments described herein are examples. Thus it will beappreciated that although the embodiments are described in terms ofspecific technologies, other technologies could be used to implementsystems.

Although example implementations of the embodiments have been depictedand described in detail herein, it will be apparent to those skilled inthe relevant art that various modifications, additions, substitutions,and the like can be made and these are therefore considered to be withinthe embodiments described herein.

Embodiments in other specific forms can be made without departing fromthe spirit or essential characteristics thereof. Certain adaptations andmodifications will be obvious to those skilled in the art. Therefore,the above discussed embodiments are considered to be illustrative andnot restrictive.

The invention claimed is:
 1. A method of identifying content of interestin a structured electronic document by an electronic device having aprocessor, an input device, and a display device, the method comprising:obtaining, by the processor, a structured electronic document from anoriginating web server; rendering the structured electronic document tothe display device; receiving through the input device at least twoseparate indications of content elements within the rendered structuredelectronic document; identifying with the processor a commoncharacteristic of the indicated content elements; and identifying anyfurther content element within the rendered structured electronicdocument sharing the common characteristic with the indicated contentelements, wherein the input device comprises a pointing device andwherein receiving the at least two separate indications of contentelements comprises receiving each indication from the pointing devicewhen applied at a displayed location of a content element in therendered structured electronic document.
 2. The method of claim 1further comprising, following receipt of the at least two separateindications, individually highlighting in the rendered structuredelectronic document each of the at least two separately indicatedcontent elements.
 3. The method of claim 1 further comprising displayinga cancel indicator adjacent a previously indicated content element forreceiving from the pointing device an indication to cancel thepreviously indicated content element.
 4. The method of claim 1 furthercomprising highlighting a content element when the pointing device islocated at the control element prior to receiving an indication of thecontent element.
 5. The method of claim 1, wherein receiving at leasttwo separate indications of content elements further comprisesidentifying a content element surrounding an indicated structuralelement.
 6. The method of claim 1, wherein receiving at least twoseparate indications of content elements further comprises offering achoice to indicate a content element from a plurality of coincidentcontent elements.
 7. The method of claim 1, wherein the input devicecomprises a keyboard and wherein receiving at least two separateindications of content elements further comprises receiving a key pressat the keyboard.
 8. The method of claim 1, further comprising displayinga text box on the display device for receiving input of a name for acontent element, and storing the inputted name.
 9. A computer programproduct comprising a computer program stored on a non-transitorycomputer-readable medium, the computer program comprising instructionswhich when executed by an electronic device having a processor, an inputdevice, and a display device cause the electronic device to carry out:obtaining, by the processor, a structured electronic document from anoriginating web server; rendering the structured electronic document tothe display device; receiving through the input device at least twoseparate indications of content elements within the rendered structuredelectronic document; identifying with the processor executing inaccordance with the stored computer program a common characteristic ofthe indicated content elements, and identifying any further contentelement within the rendered structured electronic document sharing thecommon characteristic with the indicated content elements, wherein theinput device comprises a pointing device and wherein receiving the atleast two separate indications of content elements comprises receivingeach indication from the pointing device when applied at a displayedlocation of a content element in the rendered structured electronicdocument.
 10. The computer program product of claim 9 wherein theinstructions are extensions of a browser forming part of the electronicdevice for rendering structured electronic documents to the displaydevice.
 11. The computer program product of claim 10 wherein theextensions have access to a DOM tree of the structured electronicdocument, and the extensions utilize various browser features.
 12. Anelectronic device comprising: a stored computer program, a processorexecuting in accordance with the stored computer program, an inputdevice, and a display device, wherein the computer program comprisesinstructions which when executed by the electronic device carry out:obtaining a structured electronic document from an originating webserver; rendering the structured electronic document to the displaydevice; receiving through the input device at least two separateindications of content elements within the rendered structuredelectronic document; identifying with the processor executing inaccordance with the stored computer program a common characteristic ofthe indicated content elements, and identifying any further contentelement within the rendered structured electronic document sharing thecommon characteristic with the indicated content elements, wherein theinput device comprises a pointing device and wherein receiving the atleast two separate indications of content elements comprises receivingeach indication from the pointing device when applied at a displayedlocation of a content element in the rendered structured electronicdocument.
 13. The electronic device of claim 12 wherein the instructionsare extensions to a browser.